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SUMMARY 

This study evaluated the intra- and inter-rater reliability of the Pooling score (P-score) in clinical endoscopic evaluation of severity of swal- 
lowing disorder, considering excess residue in the pharynx and larynx. The score (minimum 4 - maximum 11) is obtained by the sum of 
the scores given to the site of the bolus, the amount and ability to control residue/bolus pooling, the latter assessed on the basis of cough, 
radage, number of dry voluntary or reflex swallowing acts (< 2, 2-5, > 5). Four judges evaluated 30 short films of pharyngeal transit of 10 
solid (1/4 of a cracker), 1 1 creamy (1 tablespoon of jam) and 9 liquid (1 tablespoon of 5 cc of water coloured with methlyene blue, 1 ml in 
100 ml) boluses in 23 subjects (10 M/13 F, age from 31 to 76 yrs, mean age 58.56±1 1.76 years) with different pathologies. The films were 
randomly distributed on two CDs, which differed in terms of the sequence of the films, and were given to judges (after an explanatory ses- 
sion) at time 0, 24 hours later (time 1) and after 7 days (time 2). The inter- and intra-rater reliability of the P-score was calculated using the 
intra-class correlation coefficient (ICC; 3,k). The possibility that consistency of boluses could affect the scoring of the films was considered. 
The ICC for site, amount, management and the P-score total was found to be, respectively, 0.999, 0.997, 1.00 and 0.999. Clinical evaluation 
of a criterion of severity of a swallowing disorder remains a crucial point in the management of patients with pathologies that predispose 
to complications. The P-score, derived from static and dynamic parameters, yielded a very high correlation among the scores attributed by 
the four judges during observations carried out at different times. Bolus consistencies did not affect the outcome of the test; the analysis 
of variance, performed to verify if the scores attributed by the four judges to the parameters selected, might be influenced by the different 
consistencies of the boluses, was not significant. These initial data validate the clinical use of the P-score in the management of patients with 
deglutition disorders by a multidisciplinary team. 
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RIASSUNTO 

Questo studio valuta la variabilita inter ed intra-individuale del Pooling-score (P-score) nella valutazione clinica endoscopica della gra- 
vita della disfagia, considerando i ristagni in ipofaringe e laringe. Lo score (minimo 4 massimo 11) e ottenuto dalla somma del punteggio 
attribuito alia sede del bolo, alia quantitd ed alia capacitd di controllo del bolo residuo/ristagnato, quest' ultima valutata sulla base delle 
reazioni volontarie o riflesse di tosse, raclage, numero di atti deglutitori a vuoto (<2, 2-5, >5). Quattro giudici hanno valutato 30 brevi 
filmati di transiti faringei di 10 boli solidi ( 1/4 di cracker), 11 cremosi ( 1 cucchiaio di marmellata) e 9 liquidi ( 1 cucchiaio da 5 cc di acqua 
colorata con blu di metilene, 1 ml in 100 ml di acqua) di 23 soggetti (10M/13F fra 31-76 anni, eta media 58.56±11.76) affetti da patolo- 
gie diverse. 1 filmati, distribuiti su due CD in sequenza diversa e casuale, venivano sottoposti ai giudici (previa una sessione esplicativa) 
nel tempo 0, a distanza di 24 ore (tempo 1) e dopo 7 giorni (tempo 2). L' ajfidabilitd inter-individuale ed intra-individuale del P-score e 
statu calcolata, utilizzando I'intra-class correlation coefficient (ICC; 3,k). La possibilitd che la consistenza dei boli potesse influenzare 
il punteggio attribuito ai filmati e stata considerata. L'lCC per i parametri sede, quantita, gestione e il P-score totale e risultato essere 
rispettivamente: 0.999, 0.997, 1.00 e 0.999. La valutazione clinica di un criteria di gravita di un disordine della deglutizione resta un punto 
cruciale nella gestione di pazienti con patologie che predispongono a complicanze. II P-score, che deriva da parametri statici e dinamici, 
ha raggiunto una correlazione molto aha fra i punteggi attribuiti dai quattro giudici durante osservazioni eseguite in tempi diversi. Le 
consistenze del bolo non hanno infiuenzato I'esito del test: I'analisi della varianza, effettuata per verificare se il punteggio attribuito dai 
quattro giudici ai parametri selezionati potesse essere infiuenzato dalle diverse consistenze, e stata non significativa. Questiprimi dati ren- 
dono possibile un uso clinico del P-score nella gestione di pazienti con disturbi di deglutizione in un team multidisciplinare. 
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Introduction 

In patients with dysphagia due to different pathologies, 
predisposing to complications, the clinical severity of a 
swallowing disorder needs to be established Respiratory 
complications, related to false paths, taken by bolus or 
pooling of secretions, can arise in the short or long term ^. 
Clinical non-instrumental assessment (clinical swallow- 
ing evaluation - CSE) ^ alone can fail to identify silent 
aspirations or micro-aspirations. In at-risk conditions for 
the patient, and to define physiopathology of the swallow- 
ing disorder, instrumental assessment is essential in order 
to plan treatment. Currently, there is no instrumental tech- 
nique that clearly defines the risk of false pathways: vid- 
eo-fluoroscopic study of swallowing (VFSS) ^ and fiber- 
optic endoscopic evaluation of swallowing (FEES) * both 
yield false positives and false negatives ^. Choosing the 
instrumental method that best responds to clinical ques- 
tions ^ and the expertise of the clinician carrying out the 
procedure ' '° become crucial factors in clinical manage- 
ment of patients with deglutition disorders. 
Endoscopic assessment provides an exhaustive definition 
of anatomical details of pharyngeal and laryngeal surfac- 
es, including secretions or bolus residues that, in usual or 
unusual conditions, moisten the walls (coating) or occupy 
containment cavities (pooling, residue). Such a evaluation 
is considered a clinical criterion for severity that can be 
linked to the risk of respiratory complications Although 
to date a standardised grading of pharyngeal residue is not 
available, this criterion is taken into account when plan- 
ning treatment and management activities by a multidis- 
ciplinary team 

In previous publications pooling of materials is con- 
sidered in the broadest sense as any material that is present 
in the containment cavities of the hypopharynx and larynx, 
before and/or after the act of swallowing. The severity cri- 
terion proposed (Pooling score, P-score) (Table I) takes 
into account different parameters: 1) site: identified by 
anatomical landmarks; 2) amount: determined in a semi- 



quantitative fashion by the amount of pooling materials 
(coating, more or less than 50% of cavity containment 
capacity); 3) management: the ability of the patient to 
clear the residue. The score refers to a specific type of 
consistency and volume of the bolus, changing according 
to these parameters. The score considers the most severe 
condition for each parameter, reached after a sequence of 
boluses of the same consistency (in our department a se- 
quence of at least 3 boluses for each consistency). 
In clinical practice, the P-score may be integrated with 
other parameters of clinical assessment (CSE) that are 
more easily determined: sensation of the pharynx, patient 
collaboration and age (P-SCA score). Both the scores ex- 
press, as a numerical value, a continuum of severity that 
may be used in different ways, with correlations that still 
have to be verified. Therefore, a minimum score (P-score 
4-5) may indicate the absence of endoscopic signs of dys- 
phagia. A low score (P-score 6-7) may identify mild dys- 
phagia, a medium score (P-score 8-9) moderate dysphagia 
and a high score (P-score 10-11) severe dysphagia. 
This investigation considers the intra- and inter-rater reli- 
ability of the P-score among 4 judges with long-standing 
experience in the use of endoscopy. CSE parameters and 
the P-SCA score were not considered. 

Materials and methods 

Transits of 23 consecutive outpatients aged between 31 
and 76 years (average age 58.56 ± 11.76), and referred 
to our department complaining of difficulties in swallow- 
ing, were enrolled. Sample characteristics and diseases 
are reported in Table II. Four judges with expertise in the 
FEES procedure were elected. Three judges have at least 
10 years of experience, and one has 4 years of experience: 
they all routinely carry out at least 5 endoscopic assess- 
ments of swallowing per week. 

For some of the 23 subjects, the pharyngeal transit films 
of more than one bolus of a different consistency was re- 
corded to obtain a total number of 30 films: 10 solid bo- 



Table 1. P-score and P-SCA score. 


Pooling 


Endoscopic landmarks 






Vallecula 


1 




Marginal zone 


1 


Site 


Pyrlform sinus 


2 




Vestibule/vocal cords 


3 




Lower vocal cords 


4 




Coating 


1 


Amount 


IVlinlmum 


2 




Maximum 


3 




<2 


2 


Management 


2-5 


3 




>5 


4 



Sensation 



Presence = -1 
Absence = -i-l 



Bedside parameters 
Collaboration 



Presence = -1 
Absence = -fl 



Age (years) 



+1 
+2 
+2, 



(< 65) 

(65-75) 

(>75) 



Score 



P4-11 



P-SCA 3-1 6 
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Table II. Patient characteristics and diseases. 



Subject 


Gender 


Age 


Pathology 


1 PS 


F 


47 


Globus 


2AA 


F 


68 


Corticai ictus sequelae 


3 GA 


M 


65 


GERD 


4BA 


F 


62 


COPD 


5 CL 


F 


56 


Dermatomyositis 


6RR 


M 


67 


Corticai ictus sequelae 


7 IVilVil 


F 


64 


Laryngeal paralysis 


8 XL 


F 


38 


GERD 


9TS 


F 


64 


Neurological degenerative 


10 IVilVi2 


M 


63 


Corticai ictus sequelae 


11 QG 


M 


42 


Corea major 


12 DA 


F 


71 


Myasthenia 


13 IVIE 


M 


52 


H-N operated 


UDIVI 


F 


48 


Corticai ictus sequelae 


15 IViE 


F 


46 


Laryngeal paralysis 


16 BG 


M 


62 


Sjogren's syndrome 


17 CO 


F 


76 


Wallemberg sequaele 


18 BF 


F 


73 


Laryngeal paralysis 


19 CR 


M 


51 


Laryngeal paralysis 


20IViP 


M 


71 


COPD 


21 RF 


F 


67 


H-N operated 


22 CL 


M 


31 


Neuroiogiai degenerative 


23 SR 


M 


63 


Corticai ictus sequelae 



luses (1/4 of a cracker), 11 creamy boluses (1 spoonful of 
jam) and 9 liquid boluses (1 tablespoon with 5 cc of water 
dyed with methylene blue, 1 cc in 100 ml, according to 
the procedure used in our department). The films are short 
sequences of acts of swallowing that are prolonged until 
the bolus has been completely swallowed, or at least 5 
swallows. Film length varies from a few dozen seconds to 
no more than 180 seconds, so the complete session lasted 
no more than an hour, avoiding fatigue and a decrease in 
concentration among judges 

All the films were collected on two CD copies, respec- 
tively CDO and CDl, that differed with respect to the 
sequence of the films. Before the session, the rationale 
and application of the score were explained to the judges. 
Each judge was given a CD-test containing 4 pharyngeal 
transits of boluses with different consistencies, in subjects 
who differed by the severity of dysphagia. The application 
of the P-score was explained personally by the first judge 
at an explanatory session lasting 30 min. Judges were 
asked to watch CDO immediately thereafter (time 0), CDl 
at 24 hours later (time 1) and CDO after 7 days (time 2) in 
a different order than at time 0. 

Data were collected on a predefined sheet, reporting sep- 
arate findings for each parameter: pooling site, amount. 



management and total P-score. The different bolus con- 
sistencies used in our study between the pharyngeal tran- 
sit films tested is a bias " For this reason, we calculated 
the variance of scores attributed by judges to the param- 
eters selected. Finally, the score can also be applied at the 
beginning of the observation In the films considered, 
material pooling at the beginning of the observation was 
present in only a few cases, so it was decided not to apply 
the score at that time, but only after the transit of the bolus 
through the pharynx. 

The analysis of data obtained was carried out using SAS 
statistical software and the inter- and intra-rater reliability 
was calculated with the infra-class correlation coefficient 
(ICC; 3,k) 2°. 

Results 

The scores attributed by judges to the site, amount and 
management of the pooled material are shown in Tables 
nia, nib and IIIc. The site, apparently simpler, was modi- 
fied by both the first and fourth judges over time, reaching 
agreement at the last observation [ICC (3, k) 0.999]. The 
parameter amount was changed many times during obser- 
vations, with differences maintained among judges [ICC 
(3, k) 0.997]. The parameter management was replicated 
by all judges in all three observations [ICC (3, k) 1.000]. 
The total P-score attributed by the four judges confirmed 
that the criterion of severity established by the first judge 
was essentially shared by the other three judges [ICC (3, 
k) 0.999] (Table IV). 

The analysis of variance, performed to verify if the 
scores attributed by the four judges to the parameters se- 
lected, might have been influenced by the different con- 
sistency of the boluses, was not significant (interaction 
consistency*judge = 1.000 at times 0, 1,2, respectively, 
for the three parameters of the score and P-score total). 
Thus, the different types of consistencies did not influence 
the scores attributed by judges to the 30 films. 

Discussion 

The evaluation of secretions or bolus pooling in the phar- 
ynx end/or larynx represents an important step in the en- 
doscopic examination of swallowing, as it is closely cor- 
related with respiratory complications ".As previously 
mentioned, to date, standardised grading for pharyngeal 
material pooling is not available: the P-score could be 
used in this clinical context. 

From a physiopathological point of view, a swallowing 
disorder is the result of an imbalance between events that 
occur in the domain of space and time ^' domains in 
which vector forces guarantee the efficiency of defensive 
strategies, which protect the airways, or clear the contain- 
ment cavities of the bolus passed through them These 
events and forces can interact in different ways. 
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Table Ilia. Site (anatomical landmarks): descriptive analysis. 



Time 


JUDGE 


1 


0 


Q 




n 


w 


n 


n 


n 
u 


-f 
1 


1 0 


\ D 


1 0 


1 0 




2 


/I 

H 


c: 
0 


c 
0 


c 
0 




3 


C 
0 


c 
0 


c 
0 


c 
0 




4 


5 


5 


5 


5 


1 


1 


15 


15 


15 


15 




2 


5 


5 


5 


5 




3 


5 


5 


5 


6 




4 


5 


5 


5 


4 


2 


1 


15 


15 


15 


15 




2 


5 


5 


5 


5 




3 


5 


5 


5 


5 




4 


5 


5 


5 


5 



P-score: site 
ICC(3,k) 0.999 



Table lllb. Amount: descnptive analysis. 



JUDGE 



Time 




1 


2 


3 


4 




N 


^ 


N 


N 


0 


1 
2 


18 
8 


19 
7 


19 
7 


18 
8 




3 


4 


4 


4 


4 


1 


1 


18 


19 


19 


18 




2 


7 


7 


7 


8 




3 


5 


4 


4 


4 


2 


1 


19 


19 


19 


19 




2 


6 


7 


7 


7 




3 


5 


4 


4 


4 



P-score: amount 
ICC(3,k) 0.997 



Table lllc. Management: descriptive analysis. 



Time 






JUDGE 






1 


2 


3 


4 






^ 


N 


N 


0 


2 


8 


8 


8 


8 




3 


5 


5 


5 


5 




4 


17 


17 


17 


17 


1 


2 


8 


8 


8 


8 




3 


5 


5 


5 


5 




4 


17 


17 


17 


17 


2 


2 


8 


8 


8 


8 




3 


5 


5 


5 


5 




4 


17 


17 


17 


17 



P-score: management 
ICC(3,k) 1.000 



Time: time of observation 
N: number of observation 

108 



In the domain of space, where forces are acting, the P- 
score identifies the pathway and flow of the bolus: the 
pathway is identified by the direction along the digestive 
or respiratory tracts, as well as false route (penetration or 
aspiration); the flow is indicated by the amount of bolus 
that does not cross the pharynx while swallowing. This 
amount (expressed in a semi-quantitative manner by the 
score) represents the difference between the total amount 
of the bolus taken and the amount of bolus swallowed. 
In the domain of time, the score identifies events that oc- 
cur before or after swallowing, considering that material 
pooling after a previous swallow becomes a bolus for the 
next swallow, with a volume that is either increased or 
decreased. Thus, the subsequent swallowing act can clear 
the residue or push it below into the airway The P-score 
considers the sequence of swallowing acts in the "man- 
agement" parameter, evaluating the fate and final amount 
of a bolus that persists in the pharynx/larynx after 5 empty 
swallows, and also gives information about the reaction 
of the patient to material pooling or to airway invasion. 
The occurrence, or absence, of dry swallowing, cough or 
throat clearing, in response to the residue or penetration/ 
aspiration before, during or after swallowing, express the 
interaction between vectors and volumes. In this way, the 
number of dry swallows or clearing activities, related to 
the final amount of material pooling, can be assumed to be 
a parameter of efficiency of the entire sequence, closely 
linked to fatigue of muscular swallowing effectors. 
These preliminary considerations can help us in the inter- 
pretation of the score attributed by the judges. The ana- 
tomical parameter (Table Ilia: site) was well identified by 
the 4 judges: disagreement may have been influenced by 
the amount and site of the residue, considering that the 
most severe condition has to be selected for scoring (the 
residue in a lower anatomical site may have been cho- 
sen with respect to a larger volume, both parameters in- 
dicating greater severity, or vice versa). In this case, site 
and amount are closely linked: in fact, "amount" (Table 
Illb) created greater difficulties for the judges. It is worth 
remembering that the measurement of this parameter is 
semi-quantitative, and the four judges attributed a diver- 
sity of scores, which in some cases were modified during 
the three observations, but in other cases remained un- 
changed. A greater variability was seen when making a 
distinction between scores 2 and 3, while "coating" cre- 
ated fewer difficulties in score attribution. 
The third parameter (management) considers the effec- 
tiveness of the manoeuvre carried out to clear the residue, 
regardless of whether it was spontaneous or performed 
upon request, and in many cases it solves the dilemma 
between site and volume mentioned above. This param- 
eter (Table IIIc) was easily evaluated by all judges and 
in all observations, with no discrepancies. If we consider 
the total score (from 4 to 11), which marks the continuum 
of clinical severity (Table IV), it can be observed that ex- 



The Pooling-score (P-score): inter- and intra-rater reliability in endoscopic assessment of the severity of dysphagia 



Table IV. Severity criteria: descriptive analysis. 







JUDGE ^^^^^H 


Tims 
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0 








n 
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D 
0 


Q 
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7 
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B 
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A 
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A 
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c 
0 
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1 u 
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c. 


o 
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-1 -1 

1 1 
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A 
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7 




Q 
0 
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/I 
4 


/I 
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4 
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0 


c 
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0 




1 u 


0 

c 


0 


0 


0 

c 






1 


1 


1 


1 


2 


4 


8 


8 


8 


8 




5 


3 


3 


3 


3 




7 


6 


7 


7 


7 




8 


5 


4 


4 


4 




9 


5 


5 


5 


5 




10 


2 


2 


2 


2 




11 


1 


1 


1 


1 



P-score: total 

ICC(3,k) 0.999 

Time: time of observation 

N: number of observations 



treme scores were replicated by the four judges in all ob- 
servations, with good agreement. This was easy to iden- 
tify, and maintain judgement, for subjects without pooling 
or patients with bulky residues. Differences emerged in 
the medium classes of severity where the attribution of 
a different partial score may affect the attribution of a 
different class of clinical severity. In our work, these dif- 
ferences, sometimes replicated at the third observation, 
never reached statistical significance and can be assumed 
to be casual. 

Conclusions 

One of the most difficult tasks for clinicians and manage- 
ment team is to determine the severity of the swallowing 
disorder. This criterion very often refers to the risk for 
the lower airways to be invaded by the bolus or by ma- 
terial pooling into pharyngeal or laryngeal cavities. The 
instrumental criterion of severity (endoscopic or radio- 
logical) needs to be contextualised according to a more 
general clinical criterion of severity, which should make 



reference to the patient, considering that the non-instru- 
mental assessment (CSE) tends to underestimate the risk 
of aspiration, whereas instrumental assessment tends to 
overestimate it "Severity", in this case, becomes a rela- 
tive criterion, which is quantifiable by the parameters that 
define it. In clinical practice, aspiration is the most signifi- 
cant event that marks a swallowing disorder, although it is 
not the only one. 

Endoscopic and radiological evaluations are complemen- 
tary techniques ^, even if the former offers an optimal view 
of pharyngeal and laryngeal cavities, and particularly of 
material pooling Parameters related to instrumental se- 
verity, devised for radiological evaluation, might not be 
the best for application in the clinical endoscopic field, 
though they maintain their effectiveness in terms of inter- 
and intra-rater reliability Nonetheless, scores that can 
be easily applied in clinical practice are needed. There are 
several endoscopic scores reported in the literature, with 
severity criteria divided into 3, 4 or 5 levels. This division 
does not seem to interfere with the inter- and intra-rater 
reliability of a score The P-score, which considers 
anatomical and functional parameters, evaluates the inter- 
action between volumetric, vectorial and temporal events, 
disengaging the criterion of severity from the quantitative 
parameter alone. 

The high inter- and intra-rater reliability of the P-score 
was verified for anatomical, semi-quantitative and tempo- 
ral parameters. Variability among judges, which was not 
statistically significant, was expressed with respect to the 
"site" and "amount", whereas "management" seems more 
easily determinable. 

Despite the high reliability of the P-score shown herein, 
one critical point could be represented by the unequal sub- 
division of clinical severity in the sample, taken from con- 
secutive outpatients seen in our department. The patients 
with the most severe clinical conditions were not able to 
manage all consistencies or volumes tested by those less 
severely affected. In the former patients, for example, it 
was possible to administer only a few small amounts of 
the bolus with a creamy consistency. Nevertheless, the 
statistical analysis verifies the reliability of the judges in 
attributing a score for each parameter of the P-score, re- 
gardless of severity, which derives from the sum of the 
three parameters that determine it. This should not affect 
the clinical use of the score, which, in contrast, seems to 
be able to identify patients with small differences in se- 
verity. 

Further research is being undertaken to check the effec- 
tiveness of the P-score in clinical management of patients 
with swallowing disorders that have a different aetiology, 
as well as its usefulness in indicating variations after spe- 
cific treatment. 
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